Mandatory Image Censorship? Red Light On for Online Communities

[Anchor]

Starting next month, when users upload images to online communities, AI will inspect them first. As measures to prevent the distribution of illegal video recordings are expanded to include images, service providers such as Google, Meta, and Naver must use AI to filter images that users attempt to post to check for any issues. Backlash is growing among community users, who argue that this could lead to surveillance of private data or prior censorship.

[Reporter]

In fact, this law is rooted in the Telecommunications Business Act, which was amended as the so-called 'Nth Room Prevention Act.'

This law was amended in June 2020 following the 'Nth Room' incident to prevent the distribution of digital sex crimes.

Under this law, internet service providers are obligated to technically manage their platforms to prevent the sharing of illegal recordings.

Previously, this only applied to video files, but it has recently become a hot topic as it became known that image files will soon be included.

[Son Ji-won / Attorney at Innovation Law Firm: "The core measure is the pre-filtering system, known as identification and restriction of posting. When a user attempts to post information or content, the system identifies and filters the content before it is published. Simply put, the Korea Media and Communications Commission (KMCC) provides digital identification data of illegal recordings to internet service providers. When a user tries to upload information or content, the service providers compare it with this data, and if there is a match, the upload is blocked."]

Under this decree, service providers subject to these measures are defined as those with annual sales of 1 billion won or more in the previous year, or an average of 100,000 or more daily users. Therefore, major platforms like Google, Meta, X, and KakaoTalk Open Chat, as well as popular online communities like DC Inside and Theqoo, are all subject to the regulation.

① Prior censorship?

Some point out that this regulation amounts to prior censorship.

In fact, some organizations filed a constitutional petition claiming the decree was unconstitutional, but the Constitutional Court dismissed it in a unanimous decision, ruling that the legislative purpose was legitimate and the means were appropriate.

At the time, the Korea Media and Communications Commission (KMCC) also explained that it is far from censorship, as it merely compares the identification values of uploaded videos with those of videos already determined to be illegal.

[Son Ji-won / Attorney at Innovation Law Firm: "In the case of digital sexual violence or illegal recordings, the damage is so severe and has become a major issue in Korean society, which is why such a strong system was introduced. However, there is virtually no legislation in the world that mandates such pre-filtering in the online space. Concerns that this is a law with a 'prior censorship-like nature' seem to have resurfaced with this expanded implementation."]

② Is it effective?

[Kim Min-ho / Professor at Sungkyunkwan University Law School: "Secondly, companies still lack an image DNA database. AI filtering is only possible when there is a sufficient image DNA database, but without it, errors will occur. An error means that perfectly normal videos and images will fail to upload or be posted. If that happens, users will naturally experience inconvenience."]

③ How will the costs be covered?

Another issue being pointed out is the cost.

Once this measure is implemented, expensive GPU servers capable of AI deep learning will be required.

[Kim Min-ho / Professor at Sungkyunkwan University Law School: "The volume of images is bound to be massive. Performing technical filtering on such a huge volume of data can be a significant burden from a company's perspective."]

The KMCC stated that it plans to run a grace period until the end of the year to allow service providers to set up equipment and complete verification procedures, and will hold an online briefing session this month to support technical preparations.

[Kim Min-ho / Professor at Sungkyunkwan University Law School: "Ultimately, while this law is a policy that should be implemented for user protection, if implemented poorly, it could end up putting only domestic service providers in a difficult position and causing inconvenience to users. These are the concerns."]

[Son Ji-won / Attorney at Innovation Law Firm: "In the end, this system is bound to have a major impact on the basic rights of ordinary citizens. I hope it does not function as a stepping stone to expand censorship to other illegal information, such as copyright-infringing or defamatory content. To eradicate digital sexual violence, it is most essential to prevent it through the deterrent effect of stricter punishments."]

What are your thoughts on the AI-based image filtering system, which is scheduled to take effect this July amid much concern?
※ Please note: This article was translated by AI and may contain errors.